Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. However, their iterative refinement process in high-dimensional data space results in slow inference speed, which restricts their application in real-time systems. Previous works have explored speeding up by minimizing the number of inference steps but at the cost of sample quality. In this work, to improve the inference speed for DDPM-based TTS model while achieving high sample quality, we propose ResGrad, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model (e.g., FastSpeech 2) by predicting the residual between the model output and the corresponding ground-truth speech. ResGrad has several advantages: 1) Compare with other acceleration methods for DDPM which need to synthesize speech from scratch, ResGrad reduces the complexity of task by changing the generation target from ground-truth mel-spectrogram to the residual, resulting into a more lightweight model and thus a smaller real-time factor. 2) ResGrad is employed in the inference process of the existing TTS model in a plug-and-play way, without re-training this model. We verify ResGrad on the single-speaker dataset LJSpeech and two more challenging datasets with multiple speakers (LibriTTS) and high sampling rate (VCTK). Experimental results show that in comparison with other speed-up methods of DDPMs: 1) ResGrad achieves better sample quality with the same inference speed measured by real-time factor; 2) with similar speech quality, ResGrad synthesizes speech faster than baseline methods by more than 10 times. Audio samples are available at https://resgrad1.github.io/.

Salient object detection (SOD) aims to determine the most visually attractive objects in an image. With the development of virtual reality technology, 360{\deg} omnidirectional image has been widely used, but the SOD task in 360{\deg} omnidirectional image is seldom studied due to its severe distortions and complex scenes. In this paper, we propose a Multi-Projection Fusion and Refinement Network (MPFR-Net) to detect the salient objects in 360{\deg} omnidirectional image. Different from the existing methods, the equirectangular projection image and four corresponding cube-unfolding images are embedded into the network simultaneously as inputs, where the cube-unfolding images not only provide supplementary information for equirectangular projection image, but also ensure the object integrity of the cube-map projection. In order to make full use of these two projection modes, a Dynamic Weighting Fusion (DWF) module is designed to adaptively integrate the features of different projections in a complementary and dynamic manner from the perspective of inter and intra features. Furthermore, in order to fully explore the way of interaction between encoder and decoder features, a Filtration and Refinement (FR) module is designed to suppress the redundant information between the feature itself and the feature. Experimental results on two omnidirectional datasets demonstrate that the proposed approach outperforms the state-of-the-art methods both qualitatively and quantitatively.

Manual prescription of the field of view (FOV) by MRI technologists is variable and prolongs the scanning process. Often, the FOV is too large or crops critical anatomy. We propose a deep-learning framework, trained by radiologists' supervision, for automating FOV prescription. An intra-stack shared feature extraction network and an attention network are used to process a stack of 2D image inputs to generate output scalars defining the location of a rectangular region of interest (ROI). The attention mechanism is used to make the model focus on the small number of informative slices in a stack. Then the smallest FOV that makes the neural network predicted ROI free of aliasing is calculated by an algebraic operation derived from MR sampling theory. We retrospectively collected 595 cases between February 2018 and February 2022. The framework's performance is examined quantitatively with intersection over union (IoU) and pixel error on position, and qualitatively with a reader study. We use the t-test for comparing quantitative results from all models and a radiologist. The proposed model achieves an average IoU of 0.867 and average ROI position error of 9.06 out of 512 pixels on 80 test cases, significantly better (P<0.05) than two baseline models and not significantly different from a radiologist (P>0.12). Finally, the FOV given by the proposed framework achieves an acceptance rate of 92% from an experienced radiologist.

The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

Myocardial pathology segmentation (MyoPS) can be a prerequisite for the accurate diagnosis and treatment planning of myocardial infarction. However, achieving this segmentation is challenging, mainly due to the inadequate and indistinct information from an image. In this work, we develop an end-to-end deep neural network, referred to as MyoPS-Net, to flexibly combine five-sequence cardiac magnetic resonance (CMR) images for MyoPS. To extract precise and adequate information, we design an effective yet flexible architecture to extract and fuse cross-modal features. This architecture can tackle different numbers of CMR images and complex combinations of modalities, with output branches targeting specific pathologies. To impose anatomical knowledge on the segmentation results, we first propose a module to regularize myocardium consistency and localize the pathologies, and then introduce an inclusiveness loss to utilize relations between myocardial scars and edema. We evaluated the proposed MyoPS-Net on two datasets, i.e., a private one consisting of 50 paired multi-sequence CMR images and a public one from MICCAI2020 MyoPS Challenge. Experimental results showed that MyoPS-Net could achieve state-of-the-art performance in various scenarios. Note that in practical clinics, the subjects may not have full sequences, such as missing LGE CMR or mapping CMR scans. We therefore conducted extensive experiments to investigate the performance of the proposed method in dealing with such complex combinations of different CMR sequences. Results proved the superiority and generalizability of MyoPS-Net, and more importantly, indicated a practical clinical application.

在多种方案中，多幕科建议专门为用户检索相关项目，这在工业推荐系统中无处不在。这些方案享有用户和项目中的一部分重叠，而不同方案的分布则不同。多阶段建模的关键点是有效地最大程度地利用全幕纳罗来信息，并在多种情况下为用户和项目生成适应性表示。我们总结了三个实用挑战，这些挑战无法很好地解决多幕科建模：（1）在多种情况下缺乏细粒度和脱钩的信息传输控制。 （2）整个空间样品的开发不足。 （3）项目的多幕科代表性分解问题。在本文中，我们提出了一种情景自适应和自我监督（SASS）模型，以解决上述三个挑战。具体而言，我们使用场景自适应门单元设计了多层场景自适应转移（ML-SAT）模块，以相当细粒度且脱钩的方式选择并融合从整个场景到单个场景的有效传输信息。为了充分利用整个空间样品的功能，引入了包括预训练和微调在内的两阶段训练过程。预训练阶段是基于场景监督的对比学习任务，并从标记和未标记的数据空间中绘制的培训样本。该模型是在用户端和项目方面对称创建的，因此我们可以在不同情况下获得项目的区分表示。公共和工业数据集的广泛实验结果证明了SASS模型比最先进的方法的优越性。该模型还可以在在线A/B测试中平均每位用户的观看时间提高8.0％以上。

可以通过看不见的合作伙伴生成可以实现零拍打协调（ZSC）的代理是在合作多代理增强学习（MARL）中的新挑战。最近，一些研究通过在培训过程中将代理暴露于不同的伴侣中，从而在ZSC中取得了进展。他们通常在训练伴侣时涉及自我竞争，因为他们隐含地假设任务是同质的。但是，许多现实世界的任务都是异质的，因此以前的方法可能会失败。在本文中，我们首次研究了异质ZSC问题，并提出了一种基于协同进化的通用方法，该方法通过三个子过程进行了协调的两个代理和合作伙伴种群：配对，更新和选择。协作烹饪任务的实验结果表明需要考虑异质环境，并说明我们所提出的方法是异构合作MARL的有前途解决方案。

分割高度重叠的图像对象是具有挑战性的，因为图像上的真实对象轮廓和遮挡边界之间通常没有区别。与先前的实例分割方法不同，我们将图像形成模拟为两个重叠层的组成，并提出了双层卷积网络（BCNET），其中顶层检测到遮挡对象（遮挡器），而底层则渗透到部分闭塞实例（胶囊）。遮挡关系与双层结构的显式建模自然地将遮挡和遮挡实例的边界解散，并在掩模回归过程中考虑了它们之间的相互作用。我们使用两种流行的卷积网络设计（即完全卷积网络（FCN）和图形卷积网络（GCN））研究了双层结构的功效。此外，我们通过将图像中的实例表示为单独的可学习封闭器和封闭者查询，从而使用视觉变压器（VIT）制定双层解耦。使用一个/两个阶段和基于查询的对象探测器具有各种骨架和网络层选择验证双层解耦合的概括能力，如图像实例分段基准（可可，亲戚，可可）和视频所示实例分割基准（YTVIS，OVIS，BDD100K MOTS），特别是对于重闭塞病例。代码和数据可在https://github.com/lkeab/bcnet上找到。

尽管视频实例细分（VIS）已经取得了迅速的进步，但当前的方法难以预测具有准确边界细节的高质量面具。此外，预测的分割经常会随着时间的流逝而波动，表明时间一致性线索被忽略或不充分利用。在本文中，我们着手解决这些问题，目的是实现VIS的高度详细且更具时间稳定的面具预测。我们首先提出了视频蒙版转换方法（VMT）方法，得益于高效的视频变压器结构，能够利用细粒度的高分辨率功能。我们的VMT检测和组在视频段中每个曲目的稀疏易用错误时空区域稀疏，然后使用局部和实例级别的提示对其进行完善。其次，我们确定流行的YouTube-VIS数据集的粗边界注释构成了一个主要限制因素。因此，根据我们的VMT体系结构，我们通过迭代培训和自我纠正设计了一种自动注释细化方法。为了基准VIS的高质量掩码预测，我们介绍了HQ-YTVIS数据集，该数据集由手动重新注销的测试集和我们的自动完善培训数据组成。我们将VMT与HQ-YTVI的最新最新方法以及YouTube-VIS，OVIS和BDD100K MOTS基准进行了比较。实验结果清楚地证明了我们方法通过捕获精确的细节来分割复杂和动态对象的功效和有效性。

关于图像协调的最新作品将问题作为像素图像翻译任务通过大型自动编码器解决。在处理高分辨率图像时，它们的性能不令人满意和缓慢的推理速度。在这项工作中，我们观察到调整基本图像过滤器的输入参数，例如亮度和对比度，足以使人类从复合材料的图像中产生逼真的图像。因此，我们将图像协调作为图像级回归问题，以了解人类用于任务的过滤器的参数。我们提出了一个用于图像协调的谐波框架。与基于黑框自动编码器的先前方法不同，Harmonizer包含用于过滤器参数预测的神经网络，以及用于图像协调的几个白色框过滤器（基于预测参数）。我们还引入了级联回归器和一个动态损失策略，以使和声使更稳定地学习过滤器论点。由于我们的网络仅输出图像级参数和我们使用的过滤器是有效的，因此谐波比现有方法更轻，更快。全面的实验表明，谐波可以超过现有方法，尤其是在高分辨率输入的情况下。最后，我们将谐波应用于视频和谐，以1080p分辨率在框架和56 fps上实现一致的结果。代码和型号可在以下网址提供：https：//github.com/zhkkke/harmonizer。